Crate optimization [−] [src]
Collection of various optimization algorithms and strategies.
Building Blocks
Each central primitive is specified by a trait:
Function
- Specifies a function that can be minimizedFunction1
- Extends aFunction
by its first derivativeSummation
- Represents a summation of functions, exploited, e.g., by SGDSummation1
- Analogous toFunction
andFunction1
but forSummation
Minimizer
- A minimization algorithmEvaluation
- A function evaluationf(x) = y
that is returned by aMinimizer
Func
- A new-type wrapper for theFunction
traitNumericalDifferentiation
- Provides numerical differentiation for arbitraryFunction
s
Algorithms
Currently, the following algorithms are implemented. This list is not final and being expanded over time.
GradientDescent
- Iterative gradient descent minimization, supporting various line search methods:FixedStepWidth
- No line search is performed, but a fixed step width is usedExactLineSearch
- Exhaustive line search over a set of step widthsArmijoLineSearch
- Backtracking line search using the Armijo rule as stopping criterion
StochasticGradientDescent
- Iterative stochastic gradient descenent minimazation, currently using a fixed step width
Modules
problems |
Common optimization problems for testing purposes. |
Structs
ArmijoLineSearch |
Backtracking line search evaluating the Armijo rule at each step width. |
ExactLineSearch |
Brute-force line search minimizing the objective function over a set of step width candidates, also known as exact line search. |
FixedStepWidth |
Uses a fixed step width |
Func |
New-type to support optimization of arbitrary functions without requiring to implement a trait. |
GradientDescent |
A simple Gradient Descent optimizer. |
NumericalDifferentiation |
Wraps a function for which to provide numeric differentiation. |
StochasticGradientDescent |
Provides stochastic Gradient Descent optimization. |
Traits
Evaluation |
Captures the essence of a function evaluation. |
Function |
Defines an objective function |
Function1 |
Defines an objective function |
LineSearch |
Define a line search method, i.e., choosing an appropriate step width. |
Minimizer |
Defines an optimizer that is able to minimize a given objective function |
Summation |
Defines a summation of individual functions, i.e., f(x) = ∑ᵢ fᵢ(x). |
Summation1 |
Defines a summation of individual functions |